{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "20dde151-593c-4c8d-8a11-8f447cec913c",
   "metadata": {},
   "source": [
    "# LLMs and Pirates 🏴‍☠️🦜⚓\n",
    "The theme is of a pirate-y nature. If ye be havin' any wonderings about what a life in the seas be like, yer in the right place, matey. Arrr!\n",
    "\n",
    "This notebook demonstrates use of \n",
    "- LLM APIs: OpenAI ChatGPT, Anthropic Claude, Google Gemini\n",
    "- Gradio to build some UIs to test out the LLMs\n",
    "- Streaming mode for LLM output\n",
    "- Text-to-speech via OpenAI's TTS API\n",
    "- Image generation with DALL-E"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cfcb547-40f6-47a7-9eb7-a9fc826032ed",
   "metadata": {},
   "source": [
    "## Initialization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a07e7793-b8f5-44f4-aded-5562f633271a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import json\n",
    "import gradio as gr\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "from openai import OpenAI\n",
    "import google.generativeai\n",
    "import anthropic\n",
    "\n",
    "import random\n",
    "\n",
    "import base64\n",
    "from io import BytesIO\n",
    "from PIL import Image\n",
    "from IPython.display import Audio, display"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4e0cb743-11b6-4973-9a25-9d9a4fb9de86",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load environment variables in a file called .env\n",
    "# Print the key prefixes to help with any debugging\n",
    "\n",
    "load_dotenv(override=True)\n",
    "openai_api_key    = os.getenv('OPENAI_API_KEY')\n",
    "anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')\n",
    "google_api_key    = os.getenv('GOOGLE_API_KEY')\n",
    "\n",
    "if openai_api_key:\n",
    "    print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n",
    "else:\n",
    "    print(\"OpenAI API Key not set\")\n",
    "    \n",
    "if anthropic_api_key:\n",
    "    print(f\"Anthropic API Key exists and begins {anthropic_api_key[:7]}\")\n",
    "else:\n",
    "    print(\"Anthropic API Key not set\")\n",
    "\n",
    "if google_api_key:\n",
    "    print(f\"Google API Key exists and begins {google_api_key[:8]}\")\n",
    "else:\n",
    "    print(\"Google API Key not set\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d839e479-8ac0-4294-9f87-21ecf55a380a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialize LLMs\n",
    "openai = OpenAI()\n",
    "claude = anthropic.Anthropic()\n",
    "google.generativeai.configure()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f37fe8f8-2e2e-4988-a5ec-3ead885f0db9",
   "metadata": {},
   "source": [
    "## Simple Zero-shot Example\n",
    "Example of a single question given by the user, and we check out responses from different LLMs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "497f0265-9f7c-4bd1-a299-893623420cae",
   "metadata": {},
   "outputs": [],
   "source": [
    "system_message = 'You are a pirate. You reply to questions in a thick pirate accent.'\n",
    "user_message = 'What is a pet name you would give to your pet turtle?'\n",
    "\n",
    "# GPT 4o mini\n",
    "print('--------------------------------------------')\n",
    "messages = [\n",
    "    {'role':'system', 'content':system_message},\n",
    "    {'role':'user',   'content':user_message}\n",
    "]\n",
    "response = openai.chat.completions.create(model='gpt-4o-mini', messages=messages)\n",
    "print('GPT 4o mini:')\n",
    "print(response.choices[0].message.content)\n",
    "print('--------------------------------------------')\n",
    "\n",
    "# Claude\n",
    "print('--------------------------------------------')\n",
    "response = claude.messages.create(\n",
    "    model=\"claude-3-5-sonnet-latest\",\n",
    "    max_tokens=200,\n",
    "    temperature=0.7,\n",
    "    system=system_message,\n",
    "    messages=[\n",
    "        {\"role\": \"user\", \"content\": user_message},\n",
    "    ]\n",
    ")\n",
    "print('Claude:')\n",
    "print(response.content[0].text)\n",
    "print('--------------------------------------------')\n",
    "\n",
    "# Gemini\n",
    "print('--------------------------------------------')\n",
    "print('Gemini:')\n",
    "gemini = google.generativeai.GenerativeModel(\n",
    "    model_name='gemini-2.0-flash-exp',\n",
    "    system_instruction=system_message\n",
    ")\n",
    "response = gemini.generate_content(user_message)\n",
    "print(response.text)\n",
    "print('--------------------------------------------')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac5c2c23-7032-446f-b55c-22ab6578b36d",
   "metadata": {},
   "source": [
    "## Chat with a Pirate!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97b6ad2f-c300-42bc-8b4d-46f418b4dab9",
   "metadata": {},
   "outputs": [],
   "source": [
    "system_message = 'You are a pirate. You reply to questions in a thick pirate accent.'\n",
    "\n",
    "def chat(user_message, history):\n",
    "    messages = [{'role':'system', 'content':system_message}] + history + [{'role':'user', 'content':user_message}]\n",
    "    response = openai.chat.completions.create(model='gpt-4o-mini', messages=messages)\n",
    "    reply = response.choices[0].message.content\n",
    "    return reply\n",
    "\n",
    "gr.ChatInterface(fn=chat, type='messages').launch()\n",
    "\n",
    "# Example: \n",
    "# - What is rum made of? and why are pirates so fond of it?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8563ae0-0c49-446f-bdbd-9b1eb2fb83ff",
   "metadata": {},
   "source": [
    "## Dynamic Selection & Streaming\n",
    "Below is a sample UI with dynamic model selection. The LLM response is also returned in streaming mode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "05caeb04-e350-404d-aa67-d2cfed61ad5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "system_message = 'You are a pirate. You reply to questions in a thick pirate accent.'\n",
    "\n",
    "def stream_gpt(prompt):\n",
    "    messages = [\n",
    "        {\"role\": \"system\", \"content\": system_message},\n",
    "        {\"role\": \"user\", \"content\": prompt}\n",
    "      ]\n",
    "    stream = openai.chat.completions.create(\n",
    "        model='gpt-4o-mini',\n",
    "        messages=messages,\n",
    "        stream=True\n",
    "    )\n",
    "    result = \"\"\n",
    "    for chunk in stream:\n",
    "        result += chunk.choices[0].delta.content or \"\"\n",
    "        yield result\n",
    "\n",
    "def stream_claude(prompt):\n",
    "    result = claude.messages.stream(\n",
    "        model=\"claude-3-haiku-20240307\",\n",
    "        max_tokens=1000,\n",
    "        temperature=0.7,\n",
    "        system=system_message,\n",
    "        messages=[\n",
    "            {\"role\": \"user\", \"content\": prompt},\n",
    "        ],\n",
    "    )\n",
    "    response = \"\"\n",
    "    with result as stream:\n",
    "        for text in stream.text_stream:\n",
    "            response += text or \"\"\n",
    "            yield response\n",
    "\n",
    "def stream_model(prompt, model):\n",
    "    if model==\"GPT\":\n",
    "        result = stream_gpt(prompt)\n",
    "    elif model==\"Claude\":\n",
    "        result = stream_claude(prompt)\n",
    "    else:\n",
    "        raise ValueError(\"Unknown model\")\n",
    "    yield from result\n",
    "\n",
    "view = gr.Interface(\n",
    "    fn=stream_model,\n",
    "    inputs=[gr.Textbox(label=\"Your message:\"), gr.Dropdown([\"GPT\", \"Claude\"], label=\"Select model\", value=\"GPT\")],\n",
    "    outputs=[gr.Markdown(label=\"Response:\")],\n",
    "    flagging_mode=\"never\"\n",
    ")\n",
    "view.launch()\n",
    "# Example\n",
    "# - What's the favorite sea for a pirate to have adventures on?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ea79f38-071b-4b10-9750-01c76414d52d",
   "metadata": {},
   "source": [
    "## Voiced-out Responses\n",
    "Below is a sample UI where the LLM response is voiced out!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6800be93-857f-4cd4-9b7d-309ec3515da6",
   "metadata": {},
   "outputs": [],
   "source": [
    "def talker(message):\n",
    "    response = openai.audio.speech.create(\n",
    "        model=\"gpt-4o-mini-tts\",\n",
    "        # model=\"tts-1\",\n",
    "        voice=\"ash\",\n",
    "        instructions=\"Speak with a thick pirate accent.\",\n",
    "        input=message)\n",
    "\n",
    "    audio_stream = BytesIO(response.content)\n",
    "    output_filename = \"output_audio.mp3\"\n",
    "    with open(output_filename, \"wb\") as f:\n",
    "        f.write(audio_stream.read())\n",
    "\n",
    "    # Play the generated audio\n",
    "    display(Audio(output_filename, autoplay=True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19baac47-eaad-4e43-927c-5cc7ba27986a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# small test\n",
    "talker(\"Not all treasure is gold and silver, matey.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2d057ec5-9f0a-4e9e-8a8c-f8d60b4e490d",
   "metadata": {},
   "outputs": [],
   "source": [
    "system_message = 'You are a pirate. You reply to questions in a thick pirate accent.'\n",
    "\n",
    "def chat(user_message, history):\n",
    "    messages = [{'role':'system', 'content':system_message}] + history + [{'role':'user', 'content':user_message}]\n",
    "    response = openai.chat.completions.create(model='gpt-4o-mini', messages=messages)\n",
    "    reply = response.choices[0].message.content\n",
    "    talker(reply)\n",
    "    return reply\n",
    "\n",
    "gr.ChatInterface(fn=chat, type='messages').launch()\n",
    "\n",
    "# Example\n",
    "# - What are some of the skills necessary for life as a pirate?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba0b21e2-c7a9-43cd-9c93-b062fe944d65",
   "metadata": {},
   "source": [
    "## TOOLS\n",
    "Below is an example of incorporating the use of *tools* to augment our LLMs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f8239a8-564a-42f2-b428-8a485c99dc4c",
   "metadata": {},
   "source": [
    "### Tool 1: Pirate Quotes Generator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13f3a110-80be-466e-94a5-23db0f0bb6cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "pirate_quotes = [\n",
    "    \"May your blade always be wet, and powder dry.\",\n",
    "    \"Under a black flag we sail and the sea shall be our empire.\",\n",
    "    \"Ahoy matey! Let’s trouble the water!\",\n",
    "    \"Under a black flag we shall sail.\",\n",
    "    \"Obey the captain or learn to swim.\",\n",
    "    \"I be ruler of the seven seas!\",\n",
    "    \"Shiver me timbers!\",\n",
    "    \"Avast, ye scurvy dog.\",\n",
    "    \"Walk the plank, ye scallywag.\",\n",
    "    \"Pillage and plunder.\",\n",
    "    \"Batten down the hatches.\",\n",
    "    \"Weigh anchor and hoist the mizzen.\",\n",
    "    \"Here be treasure, matey.\",\n",
    "    \"Thar she blows!\",\n",
    "    \"Yo ho ho and a bottle of rum.\",\n",
    "    \"Prepare to be boarded.\",\n",
    "    \"May your anchor be tight, your cork be loose, your rum be spiced and your compass be true.\",\n",
    "    \"Loot is first and wimmen second. Because if ye have the first ye’ll have the second, but if ye have the second ye won’t have the first for long!\",\n",
    "    \"The rougher the seas, the smoother we sail. Ahoy!\",\n",
    "    \"No cause is lost if there is but one fool left to fight for it.\",\n",
    "    \"Avast ye landlubbers! Ye can throw ye lunch in Davy Jones’ locker, but not yer homework!\"\n",
    "]\n",
    "\n",
    "def pirate_quoter(num_quotes):\n",
    "    print('-----------------------------------------')\n",
    "    print(f\"{num_quotes} pirate quote(s) requested\")\n",
    "    selected_pirate_quotes = random.sample(pirate_quotes, num_quotes)\n",
    "    print(selected_pirate_quotes)\n",
    "    print('-----------------------------------------')\n",
    "    return selected_pirate_quotes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2bdd166-3b24-4dc9-9890-c568cd99e6b3",
   "metadata": {},
   "source": [
    "### Tool 2: Pet Namer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5b3b54de-f7c8-405b-8c09-2f0bd6d4a537",
   "metadata": {},
   "outputs": [],
   "source": [
    "def pet_namer(animal):\n",
    "    print('-----------------------------------------')\n",
    "    print(f\"Name for a pet {animal} requested\")\n",
    "    messages = [\n",
    "        {'role':'system', 'content':'You are a pirate. You will be given an animal, and you will decide on some pet names that you may give it if it was your pet. You will reply with only the pet names for this animal, separated by commas.'},\n",
    "        {'role':'user',   'content':animal}\n",
    "    ]\n",
    "    response = openai.chat.completions.create(model='gpt-4o-mini', messages=messages)\n",
    "    reply = response.choices[0].message.content\n",
    "    suggested_pet_names = reply.replace('and ','').split(', ')\n",
    "    print(suggested_pet_names)\n",
    "    print('-----------------------------------------')\n",
    "    return suggested_pet_names"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f94f3a97-a50d-4ae7-9e46-506db1b638bc",
   "metadata": {},
   "source": [
    "### Tools in Action"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0a2e94c-717e-49c4-ad6b-a1005c886462",
   "metadata": {},
   "outputs": [],
   "source": [
    "pirate_quoter_tool = {\n",
    "    \"name\": \"pirate_quoter\",\n",
    "    \"description\": \"Get random pirate quotes. Call this whenever you are asked for pirate quotes.\",\n",
    "    \"parameters\": {\n",
    "        \"type\": \"object\",\n",
    "        \"properties\": {\n",
    "            \"num_quotes\": {\n",
    "                \"type\": \"integer\",\n",
    "                \"description\": \"The number of pirate quotes requested by the user.\",\n",
    "            },\n",
    "        },\n",
    "        \"required\": [\"num_quotes\"],\n",
    "        \"additionalProperties\": False\n",
    "    }\n",
    "}\n",
    "\n",
    "pet_namer_tool = {\n",
    "    \"name\": \"pet_namer\",\n",
    "    \"description\": \"Get a pet name for a given animal. Call this whenever you are asked to suggest a name for a pet.\",\n",
    "    \"parameters\": {\n",
    "        \"type\": \"object\",\n",
    "        \"properties\": {\n",
    "            \"animal\": {\n",
    "                \"type\": \"string\",\n",
    "                \"description\": \"The kind of animal for which to suggest a pet name.\",\n",
    "            },\n",
    "        },\n",
    "        \"required\": [\"animal\"],\n",
    "        \"additionalProperties\": False\n",
    "    }\n",
    "}\n",
    "\n",
    "tools = [\n",
    "    {\"type\": \"function\", \"function\": pirate_quoter_tool}, \n",
    "    {\"type\": \"function\", \"function\": pet_namer_tool}\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c87fd2c-1015-442f-8b09-627083e4cfc5",
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(message, history):\n",
    "    messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model='gpt-4o-mini', messages=messages, tools=tools)\n",
    "    \n",
    "    # handle tool calls\n",
    "    if response.choices[0].finish_reason==\"tool_calls\":\n",
    "        # add response to list of messages for upcoming API call (at the end of this if)\n",
    "        message = response.choices[0].message\n",
    "        messages.append(message)\n",
    "        \n",
    "        # loop over scheduled tool calls\n",
    "        tool_calls = response.choices[0].message.tool_calls\n",
    "        for i in range(len(tool_calls)):\n",
    "            tool_call_i = tool_calls[i]\n",
    "            tool_func_i = tool_call_i.function\n",
    "            tool_name_i = tool_func_i.name\n",
    "            tool_args_i = json.loads(tool_func_i.arguments)\n",
    "            if tool_name_i == 'pirate_quoter':\n",
    "                num_quotes = tool_args_i.get('num_quotes')\n",
    "                response = {\n",
    "                    \"role\": \"tool\",\n",
    "                    \"content\": json.dumps({\"num_quotes\": num_quotes, \"quotes\": pirate_quoter(num_quotes)}),\n",
    "                    \"tool_call_id\": tool_call_i.id\n",
    "                }\n",
    "            elif tool_name_i == 'pet_namer':\n",
    "                animal = tool_args_i.get('animal')\n",
    "                response = {\n",
    "                    \"role\": \"tool\",\n",
    "                    \"content\": json.dumps({\"animal\": animal, \"pet_names\": pet_namer(animal)}),\n",
    "                    \"tool_call_id\": tool_call_i.id\n",
    "                }\n",
    "            messages.append(response)\n",
    "\n",
    "        # finalize tool calls\n",
    "        response = openai.chat.completions.create(model='gpt-4o-mini', messages=messages)\n",
    "\n",
    "    # return\n",
    "    reply = response.choices[0].message.content\n",
    "    return reply\n",
    "\n",
    "gr.ChatInterface(fn=chat, type=\"messages\").launch()\n",
    "\n",
    "# example tool-related requests you may ask during the conversation\n",
    "# - give me 3 pirate quotes\n",
    "# - What is a pet name you would give to your pet turtle?\n",
    "# - What are some pet names you may give to your pet monkey?\n",
    "# - give me 5 pirate quotes and then give me a couple of names you may choose for your pet seagull"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8cef4d22-1308-42d9-b072-94dd3b3038c4",
   "metadata": {},
   "source": [
    "## Generating Images with DALL-E\n",
    "Now let's generate an image of a pirate with his/her favorite pet!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca49681c-08b9-4f52-8fef-b57bdec2b735",
   "metadata": {},
   "outputs": [],
   "source": [
    "def artist(animal):\n",
    "    image_response = openai.images.generate(\n",
    "            model=\"dall-e-3\",\n",
    "            prompt=f\"An image of a pirate with his or her favorite pet {animal}, in a classic watercolor painting style\",\n",
    "            size=\"1024x1024\",\n",
    "            n=1,\n",
    "            response_format=\"b64_json\",\n",
    "        )\n",
    "    image_base64 = image_response.data[0].b64_json\n",
    "    image_data = base64.b64decode(image_base64)\n",
    "    return Image.open(BytesIO(image_data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63b7d80c-e843-4662-857c-e9f4512e980d",
   "metadata": {},
   "outputs": [],
   "source": [
    "artist('monkey')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1c1b981-6327-415f-a1be-6f05f498e329",
   "metadata": {},
   "source": [
    "## The Grande Finale\n",
    "Now, we bring everything together in one grand Gradio app!\n",
    "\n",
    "The code below is self-contained and doesn't require running of any cells above. The code below uses the OpenAI LLMs only. \n",
    "\n",
    "Now, enjoy the grand finale!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aad0ff50-fee8-4332-a9f5-71d99fa59b8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import json\n",
    "import gradio as gr\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "from openai import OpenAI\n",
    "import google.generativeai\n",
    "import anthropic\n",
    "\n",
    "import random\n",
    "\n",
    "import base64\n",
    "from io import BytesIO\n",
    "from PIL import Image\n",
    "from IPython.display import Audio, display\n",
    "\n",
    "load_dotenv(override=True)\n",
    "openai_api_key = os.getenv('OPENAI_API_KEY')\n",
    "openai = OpenAI()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0d8d899-3806-4ca5-84a9-15a0212e8644",
   "metadata": {},
   "outputs": [],
   "source": [
    "pirate_quotes = [\n",
    "    \"May your blade always be wet, and powder dry.\",\n",
    "    \"Under a black flag we sail and the sea shall be our empire.\",\n",
    "    \"Ahoy matey! Let’s trouble the water!\",\n",
    "    \"Under a black flag we shall sail.\",\n",
    "    \"Obey the captain or learn to swim.\",\n",
    "    \"I be ruler of the seven seas!\",\n",
    "    \"Shiver me timbers!\",\n",
    "    \"Avast, ye scurvy dog.\",\n",
    "    \"Walk the plank, ye scallywag.\",\n",
    "    \"Pillage and plunder.\",\n",
    "    \"Batten down the hatches.\",\n",
    "    \"Weigh anchor and hoist the mizzen.\",\n",
    "    \"Here be treasure, matey.\",\n",
    "    \"Thar she blows!\",\n",
    "    \"Yo ho ho and a bottle of rum.\",\n",
    "    \"Prepare to be boarded.\",\n",
    "    \"May your anchor be tight, your cork be loose, your rum be spiced and your compass be true.\",\n",
    "    \"Loot is first and wimmen second. Because if ye have the first ye’ll have the second, but if ye have the second ye won’t have the first for long!\",\n",
    "    \"The rougher the seas, the smoother we sail. Ahoy!\",\n",
    "    \"No cause is lost if there is but one fool left to fight for it.\",\n",
    "    \"Avast ye landlubbers! Ye can throw ye lunch in Davy Jones’ locker, but not yer homework!\"\n",
    "]\n",
    "\n",
    "def pirate_quoter(num_quotes):\n",
    "    print('-----------------------------------------')\n",
    "    print(f\"{num_quotes} pirate quote(s) requested\")\n",
    "    selected_pirate_quotes = random.sample(pirate_quotes, num_quotes)\n",
    "    print(selected_pirate_quotes)\n",
    "    print('-----------------------------------------')\n",
    "    return selected_pirate_quotes\n",
    "\n",
    "def pet_namer(animal):\n",
    "    print('-----------------------------------------')\n",
    "    print(f\"Name for a pet {animal} requested\")\n",
    "    messages = [\n",
    "        {'role':'system', 'content':'You are a pirate. You will be given an animal, and you will decide on some pet names that you may give it if it was your pet. You will reply with only the pet names for this animal, separated by commas.'},\n",
    "        {'role':'user',   'content':animal}\n",
    "    ]\n",
    "    response = openai.chat.completions.create(model='gpt-4o-mini', messages=messages)\n",
    "    reply = response.choices[0].message.content\n",
    "    suggested_pet_names = reply.replace('and ','').split(', ')\n",
    "    print(suggested_pet_names)\n",
    "    print('-----------------------------------------')\n",
    "    return suggested_pet_names\n",
    "\n",
    "pirate_quoter_tool = {\n",
    "    \"name\": \"pirate_quoter\",\n",
    "    \"description\": \"Get random pirate quotes. Call this whenever you are asked for pirate quotes.\",\n",
    "    \"parameters\": {\n",
    "        \"type\": \"object\",\n",
    "        \"properties\": {\n",
    "            \"num_quotes\": {\n",
    "                \"type\": \"integer\",\n",
    "                \"description\": \"The number of pirate quotes requested by the user.\",\n",
    "            },\n",
    "        },\n",
    "        \"required\": [\"num_quotes\"],\n",
    "        \"additionalProperties\": False\n",
    "    }\n",
    "}\n",
    "\n",
    "pet_namer_tool = {\n",
    "    \"name\": \"pet_namer\",\n",
    "    \"description\": \"Get a pet name for a given animal. Call this whenever you are asked to suggest a name for a pet.\",\n",
    "    \"parameters\": {\n",
    "        \"type\": \"object\",\n",
    "        \"properties\": {\n",
    "            \"animal\": {\n",
    "                \"type\": \"string\",\n",
    "                \"description\": \"The kind of animal for which to suggest a pet name.\",\n",
    "            },\n",
    "        },\n",
    "        \"required\": [\"animal\"],\n",
    "        \"additionalProperties\": False\n",
    "    }\n",
    "}\n",
    "\n",
    "tools = [{\"type\": \"function\", \"function\": pirate_quoter_tool}, {\"type\": \"function\", \"function\": pet_namer_tool}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a99561c-2f39-4a8b-95f8-271c91afadf9",
   "metadata": {},
   "outputs": [],
   "source": [
    "def talker(message):\n",
    "    response = openai.audio.speech.create(\n",
    "        model=\"gpt-4o-mini-tts\",\n",
    "        # model=\"tts-1\",\n",
    "        voice=\"ash\",\n",
    "        instructions=\"Speak with a thick pirate accent.\",\n",
    "        input=message)\n",
    "\n",
    "    audio_stream = BytesIO(response.content)\n",
    "    output_filename = \"output_audio.mp3\"\n",
    "    with open(output_filename, \"wb\") as f:\n",
    "        f.write(audio_stream.read())\n",
    "\n",
    "    # Play the generated audio\n",
    "    display(Audio(output_filename, autoplay=True))\n",
    "\n",
    "def artist(animal):\n",
    "    image_response = openai.images.generate(\n",
    "            model=\"dall-e-3\",\n",
    "            prompt=f\"An image of a pirate with his or her favorite pet {animal}, in a classic watercolor painting style\",\n",
    "            size=\"1024x1024\",\n",
    "            n=1,\n",
    "            response_format=\"b64_json\",\n",
    "        )\n",
    "    image_base64 = image_response.data[0].b64_json\n",
    "    image_data = base64.b64decode(image_base64)\n",
    "    return Image.open(BytesIO(image_data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e37ebb53-6736-48a7-a15a-e315e62ade93",
   "metadata": {},
   "outputs": [],
   "source": [
    "system_message = 'You are a pirate. You reply to questions in a thick pirate accent.'\n",
    "def chat(history, pirate_pet_image):\n",
    "    messages = [{\"role\": \"system\", \"content\": system_message}] + history\n",
    "    response = openai.chat.completions.create(model='gpt-4o-mini', messages=messages, tools=tools)\n",
    "    # pirate_pet_image = None\n",
    "    \n",
    "    # handle tool calls\n",
    "    if response.choices[0].finish_reason==\"tool_calls\":\n",
    "        # add response to list of messages for upcoming API call (at the end of this if)\n",
    "        message = response.choices[0].message\n",
    "        messages.append(message)\n",
    "        \n",
    "        # loop over scheduled tool calls\n",
    "        tool_calls = response.choices[0].message.tool_calls\n",
    "        for i in range(len(tool_calls)):\n",
    "            tool_call_i = tool_calls[i]\n",
    "            tool_func_i = tool_call_i.function\n",
    "            tool_name_i = tool_func_i.name\n",
    "            tool_args_i = json.loads(tool_func_i.arguments)\n",
    "            if tool_name_i == 'pirate_quoter':\n",
    "                num_quotes = tool_args_i.get('num_quotes')\n",
    "                response = {\n",
    "                    \"role\": \"tool\",\n",
    "                    \"content\": json.dumps({\"num_quotes\": num_quotes, \"quotes\": pirate_quoter(num_quotes)}),\n",
    "                    \"tool_call_id\": tool_call_i.id\n",
    "                }\n",
    "            elif tool_name_i == 'pet_namer':\n",
    "                animal = tool_args_i.get('animal')\n",
    "                response = {\n",
    "                    \"role\": \"tool\",\n",
    "                    \"content\": json.dumps({\"animal\": animal, \"pet_names\": pet_namer(animal)}),\n",
    "                    \"tool_call_id\": tool_call_i.id\n",
    "                }\n",
    "                pirate_pet_image = artist(animal)\n",
    "            messages.append(response)\n",
    "\n",
    "        # finalize tool calls\n",
    "        response = openai.chat.completions.create(model='gpt-4o-mini', messages=messages)\n",
    "\n",
    "    # return\n",
    "    reply = response.choices[0].message.content\n",
    "    talker(reply)\n",
    "    history += [{\"role\":\"assistant\", \"content\":reply}]\n",
    "    return history, pirate_pet_image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0fe45e01-78b2-432b-b034-6b91daffedd4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# More involved Gradio code as we're not using the preset Chat interface!\n",
    "with gr.Blocks() as ui:\n",
    "    with gr.Row():\n",
    "        chatbot = gr.Chatbot(height=500, type=\"messages\")\n",
    "        image_output = gr.Image(height=500)\n",
    "    with gr.Row():\n",
    "        entry = gr.Textbox(label=\"Chat with our AI Assistant:\")\n",
    "    with gr.Row():\n",
    "        clear = gr.Button(\"Clear\")\n",
    "\n",
    "    # last_image = gr.State(None)  # State to store the last valid image\n",
    "\n",
    "    def do_entry(message, history):\n",
    "        history += [{\"role\":\"user\", \"content\":message}]\n",
    "        return \"\", history\n",
    "\n",
    "    entry.submit(\n",
    "        do_entry, inputs=[entry, chatbot], outputs=[entry, chatbot]\n",
    "    ).then(\n",
    "        chat, inputs=[chatbot, image_output], outputs=[chatbot, image_output]\n",
    "    )\n",
    "\n",
    "    clear.click(lambda: None, inputs=None, outputs=chatbot, queue=False)\n",
    "\n",
    "# Passing in inbrowser=True in the last line will cause a Gradio window to pop up immediately.\n",
    "ui.launch(inbrowser=True)\n",
    "\n",
    "# Examples:\n",
    "# - Who is Davey Jones?\n",
    "# - What are some animals that are usually taken as pets by pirates?\n",
    "# - I am thinking of getting myself a pet dog. What should I name him?\n",
    "# - Give me some pirate quotes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8de15a2c-db58-49da-b6de-4f048211566c",
   "metadata": {},
   "source": [
    "-------\n",
    "\n",
    "# THE END\n",
    "We leave it up to the reader as exercises to incorporate the use of tools and voice/image generation with Anthropic Claude and Google Gemini. You may also extend the above UI by adding a dropdown from which the user may select a particular voice for the `talker()` function.\n",
    "\n",
    "Finally, an addition to this notebook that I would love to see is coming up with a more authentic pirate-accented voice output. Do let me know if you figure it out!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
